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ABSTRACT 

The  development  of  metrics  for  the  comparison  of  data  obtained  from  measured  synthetic  aperture 
sonar  (SAS)  data  or  from  numerical  models  is  investigated.  Metrics  are  needed  for  quantitative 
comparisons  for  signals  obtained  from  either  different  models,  the  same  model  with  omitted  physics, 
controlled  experiments,  or  actual  operational  MCM  systems.  Specifically,  candidate  metrics  for 
model-model  comparisons  are  examined  here  with  a  goal  to  consider  raw  data  prior  to  its  reduction  to 
data  products,  which  may  be  suitable  for  input  to  classification  schemes.  The  investigated  metrics  are 
then  applied  to  model-data  comparisons. 

INTRODUCTION 

Metrics  for  quantitative  comparison  of  acoustic  color  templates  were  considered  under  ONR 
Award  N00014- 14- 1-0288.  An  acoustic  color  (AC)  template  displays  target  strength  as  a  function  of 
frequency  and  target-centered  aspect  angle.  In  that  research,  it  was  found  that  the  maximum  value  of 
the  magnitude  of  a  simple  2D  cross-correlation  of  two  AC  templates  appeared  to  be  an  adequate  metric 
for  model-model  comparisons  in  the  absences  of  injected  noise.  Model-data  comparisons  suggested 
additional  metrics  were  needed  due  to  uncertainties  within  the  measurements  (e.g.,  signal-to-noise 
ratio)  or  due  to  the  acquisition  of  only  a  limited  section  of  a  full  360°  acoustics  color  template.  During 
FY15,  target  isolation  techniques  were  applied  to  the  TREX13  data,  which  led  to  isolated  target 
signatures  for  more  than  25  targets.  For  each  target,  the  isolated  signatures  were  combined  to  form  a 
pseudo-circular  SAS  data  and  an  ultimately  AC  template.  Instead  of  working  with  these  AC  templates, 
the  current  research  worked  directly  with  isolated  target  signatures. 

The  metrics  considered  here  are  based  on  an  extension  of  ParsevaTs  theorem  to  comparisons 
between  signals  from  possibly  different  targets  or  the  same  target  in  different  orientations.  Fet  a  time- 
dependent  isolated  scattered  signal  be  denoted  by  Sj(t)  with  time  t.  Its  frequency-domain  Fourier 
transform  is  5) (/)  =  F{s,  (t)}  with  frequency  /.  The  subscript  here  denotes  the  ith  signal  from  a 
collection  of  scattered  signals.  The  total  power  in  a  signal  is  the  same  whether  computed  in  the  time 
domain  or  the  frequency  domain.  ParsevaTs  theorem  states  that  a  signal  satisfies 

=  CjSAf) |2  df.  (1) 

To  exploit  Eq.  (1),  it  is  convenient  to  write  |Sj|2  =  |Sj||Sj|  =  s^s*  where  the  last  equality  introduces  the 
complex  conjugate  s*  (and  likewise  for  the  right-hand-side  of  Eq.  (1)).  Rewriting  Eq.  (1),  where  the 
integrand  of  is  replaced  by  its  complex  product,  one  has 

O<(tK(0A  =  (2) 
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Now,  consider  a  possibly  second  signal  and  define  die  “pseudo-”  total  power  defined  as 


(Uj)  = 

<■ i,j)  =  C00Si(f)S;(f)df ■ 


(3) 

(4) 


If  i  is  the  same  as  j,  then  Eqs.  (3)  and  (4)  reduce  to  Parseval’s  theorem.  To  compare  (i,j),  to  the 
results  of  cross-correlation  corr^sit  Sy)  =  define  the  normalized  result 


[i.j\ 


< i,j ) 

JiUWjj 


(5) 


where  the  normalized  cross-correlation  is  similar  to  Eq.  (5)  with  (i,j)  understood  to  be  the  maximum 
value  in  the  magnitude  of  an  inverse  Fourier  transform  of  the  product  of  5)(/)  and  SJ  (/). 


To  examine  the  consequences  of  Eqs.  (3)  and  (4),  the  target-in-the-environment  response  (TIER) 
model  is  used  to  simulate  the  scattering  from  targets  under  a  number  of  conditions.  The  environment 
is  taken  to  be  that  of  TREX13  with  water  above  a  medium-sand  sediment.  The  density  and  sound 
speed  of  water  are  p  —  1000  kg/m3  and  c  —  1530  m/s;  while,  the  sediment’s  density,  sound  speed, 
and  loss  parameter  are  p  —  2000  kg/m3,  c  —  1694  m/s,  and  8  —  0.008,  respectively.  The  source  and 
receiver  are  co-located  at  z  =  4  m  above  the  water- sediment  boundary,  and  targets  were  placed  at 
horizontal  ranges  of  10  and  12  m.  The  signal  transmitted  by  the  source  is  a  linear- frequency- 
modulated  (LFM)  chirp  with  a  16  kHz  carrier  frequency  and  30  kHz  of  bandwidth.  The  magnitudes  of 
the  pulse-compressed  scattered  signal  are  displayed  in  Figure  1  where  the  targets  are  a  solid  aluminum 
sphere  and  cylinder.  The  radii  of  the  targets  are  0.1524  m  and  the  cylinder  is  0.61  m  in  length.  The 
source,  receiver,  and  cylinder  are  oriented  to  give  broadside  scattering.  TIER  simulations  were 
performed  for  free-field  scattering  where  only  the  direct  path  (i.e.,  in  the  TIER  Path  1)  is  included  and 
for  proud  targets  where  four  ray  paths  are  included.  Thus,  eight  signals  were  generated. 


To  facilitate  discussion  of  these  signals,  a  labeling  scheme  is  required.  For  signals  associated  with 
the  spherical  or  cylindrical  targets,  the  initial  letter  in  the  label  is  either  “S”  or  “C”.  The  next  number 
indicates  the  number  of  paths  included  in  a  simulation;  for  example,  “SI”  is  for  the  spherical  target  and 
only  direct  path  scattering.  Finally,  the  last  three  characters  indicate  the  horizontal  range  to  the  target, 
e.g.,  “hlO”.  These  labels  are  used  in  the  following  tables. 


Table  1  contains  the  values  computed  from  Eq.  (5)  where  the  cross-correlations  of  the  signals  are 
computed.  Entries  in  the  table  are  color-coded  to  aid  in  discussing  the  results.  First,  the  diagonal 
entries  are  unity  as  expected  from  the  normalization  imposed  by  Eq.  (5).  The  largest  non-diagonal 
entries  (red)  occur  for  scattering  in  the  free-field,  where  either  the  spherical  or  cylindrical  target  is 
simply  translated  by  2  meters.  An  interesting  result  is  that  the  scattered  signal  from  the  proud  spherical 
target  at  10  m  (S4hl0)  and  at  12  m  (S4hl2)  appear  to  have  fairly  high  correlations  with  the  cylindrical 
target  at  the  same  distance  (green).  This  is  a  consequence  of  the  radii  of  the  targets  and  the  cylinder  is 
at  a  broadside  orientation,  which  means  the  phase  delays  for  paths  2,  3,  and  4  are  identical  for  the 
sphere  and  cylinder. 
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SlhlO 

Slhl2 

S4hl0 

S4hl2 

ClhlO 

Clhl2 

C4hl0 

C4hl2 

SlhlO 

1.000 

0.993 

0.541 

0.490 

0.745 

0.741 

0.436 

0.447 

Slhl2 

1.000 

0.535 

0.488 

0.738 

0.745 

0.437 

0.458 

S4hl0 

1.000 

0.550 

0.768 

0.774 

0.822 

0.476 

S4hl2 

1.000 

0.644 

0.637 

0.571 

0.828 

ClhlO 

1.000 

0.991 

0.742 

0.650 

Clhl2 

1.000 

0.748 

0.637 

C4hl0 

1.000 

0.633 

C4hl2 

1.000 

Table  1.  Values  estimated  from  Eq.  (5)  where  cross-correlations  are  computed.  By  construction,  the 
table  is  symmetric,  so  lower  triangular  elements  are  omitted.  The  direct  path  scattering  from  the 
spherical  or  cylindrical  target  at  a  10  m  shows  a  high  cross-correlation  (red)  with  the  same  target  at 
its  12  m  horizontal  range. 

Table  2  is  evaluated  from  Eqs.  (3)  and  (5)  with  the  additional  constraint  that  only  real  signals  are  used, 
i.e.,  SisJ  —  9?(Si)9l(s,-).  That  is,  the  complex  conjugate  of  the  signal  is  not  used.  In  Table  2,  the  red 
entries  highlight  very  small  numbers.  These  can  easily  be  understood  from  inspection  of  Figure  1, 
which  shows  that  any  scattered  signal  for  a  target  at  10  m  is  nearly  quiescent  after  2  ms  and  hence  its 
multiplication  with  a  signal  for  a  target  at  12  m  is  negligible. 


SlhlO 

Slhl2 

S4hl0 

S4hl2 

ClhlO 

Clhl2 

C4hl0 

C4hl2 

SlhlO 

1.000 

0.002 

0.541 

0.003 

0.628 

0.002 

0.285 

0.002 

Slhl2 

1.000 

0.002 

0.411 

0.007 

0.628 

0.000 

0.179 

S4hl0 

1.000 

0.000 

0.318 

0.003 

0.657 

0.003 

S4hl2 

1.000 

0.003 

0.217 

0.001 

0.640 

ClhlO 

1.000 

0.002 

0.456 

0.014 

Clhl2 

1.000 

0.005 

0.361 

C4hl0 

1.000 

0.008 

C4hl2 

1.000 

Table  2.  Values  estimated  from  Eq.  (5)  where  Eq.  (3)  is  used  for  (i,j).  By  construction,  the  table  is 
symmetric,  so  the  lower  triangular  elements  are  omitted.  The  light  blue  entries  are  a  result  that  the 
direct  path  scattering  align  and  thus  are  re-enforced  through  the  multiplication. 

Table  3  is  evaluated  from  Eqs.  (3)  and  (5)  with  the  additional  constraint  that  the  magnitudes  of  the 
signals  are  used.  That  is,  the  complex  conjugate  of  the  signal  is  not  used.  In  Table  3,  the  red  entries 
again  highlight  small  numbers.  These  can  easily  be  understood  from  inspection  of  Figure  1,  which 
shows  that  any  scattered  signal  for  a  target  at  10  m  is  nearly  quiescent  after  2  ms  and  hence  its 
multiplication  with  a  signal  for  a  target  at  12  m  is  nearly  zero.  Note  the  larger  values  observed  in 
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Table  3  than  in  Table  2  are  a  result  of  the  conversion  of  the  integrals  to  discrete  summations  where  the 
summands  for  Table  3  are  always  positive  while  some  of  the  summands  for  Table  2  are  negative. 


SlhlO 

Slhl2 

S4hl0 

S4hl2 

ClhlO 

Clhl2 

C4hl0 

C4hl2 

SlhlO 

1.000 

0.019 

0.630 

0.018 

0.753 

0.019 

0.488 

0.018 

Slhl2 

1.000 

0.018 

0.592 

0.024 

0.756 

0.021 

0.463 

S4hl0 

1.000 

0.018 

0.472 

0.017 

0.775 

0.019 

S4hl2 

1.000 

0.022 

0.446 

0.019 

0.787 

ClhlO 

1.000 

0.022 

0.559 

0.028 

Clhl2 

1.000 

0.021 

0.494 

C4hl0 

1.000 

0.020 

C4hl2 

1.000 

Table  3.  Values  estimated  from  Eq.  (5)  where  Eq.  (3)  is  used  for  (i,j).  By  construction,  the  table  is 
symmetric,  so  the  lower  triangular  elements  are  omitted.  The  light  blue  entries  are  a  result  that  direct 
path  scattering  align  and  thus  are  re-enforced  through  the  multiplication  of  the  magnitudes. 

Table  4  is  evaluated  from  Eqs.  (3)  and  (5).  Here,  the  complex  conjugate  of  the  signal  is  used,  so  Table 
4  represents  the  time-domain  ParsevaTs  theorem  result.  In  Table  4,  the  red  entries  again  highlight 
small  numbers.  These  can  easily  be  understood  from  inspection  of  Figure  1,  which  shows  that  any 
scattered  signal  for  a  target  at  10  m  is  nearly  quiescent  after  2  ms  and  hence  its  multiplication  with  a 
signal  for  a  target  at  12  m  is  nearly  zero.  The  discrete  summations  from  the  conversion  of  the  integral 
in  Eq.  (3)  is  a  coherent  summation  of  complex  summands.  The  scattering  geometries  for  the  sphere 
and  cylinder  were  constructed  to  give  essentially  the  same  path  lengths  for  all  paths,  and  the  broadside 
orientation  of  the  cylinder  yields  a  specular  scattering  return  that  is  similar  to  the  specular  return  from 


SlhlO 

Slhl2 

S4hl0 

S4hl2 

ClhlO 

Clhl2 

C4hl0 

C4hl2 

SlhlO 

1.000 

0.024 

0.680 

0.022 

0.911 

0.025 

0.600 

0.022 

Slhl2 

1.000 

0.022 

0.627 

0.029 

0.911 

0.025 

0.554 

S4hl0 

1.000 

0.022 

0.599 

0.022 

0.941 

0.023 

S4hl2 

1.000 

0.031 

0.543 

0.025 

0.951 

ClhlO 

1.000 

0.025 

0.609 

0.032 

Clhl2 

1.000 

0.024 

0.531 

C4hl0 

1.000 

0.025 

C4hl2 

1.000 

Table  4.  Values  estimated  from  Eq.  (5)  where  Eq.  (3)  is  used  for  (i,j).  By  construction,  the  table  is 
symmetric,  so  the  lower  triangular  elements  are  omitted.  The  light  blue  entries  are  a  result  that  direct 
path  align  and  thus  are  re-enforced  through  the  multiplication  of  the  magnitudes. 
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the  sphere.  In  addition,  the  dominate  surface  elastic  wave  is  associated  with  a  Rayleigh  wave 
resonance  in  the  frequency  range  considered,  and  repeated  circumnavigations  of  the  surface  wave  on 
the  sphere  and  cylinder  traverse  a  similar  path  length.  Hence,  the  coherent  processing  enhances  the 
comparisons  of  the  scattering  from  the  sphere  and  cylinder,  which  leads  to  the  blue  entries  in  Table  4. 

The  final  result  for  the  frequency-domain  Parseval’s  theorem  is  computed  from  Eqs.  (4)  and  (5), 
and  are  tabulated  in  Table  5.  One  immediately  sees  that  many  of  the  entries  exceeds  0.900.  This  is 
simply  a  manifestation  of  Eq.  (5)  summing  up  the  acoustic  energy  in  all  frequencies  without  using 
phase  information;  whereas  the  cross-correlation  results  in  Table  1  use  the  maximum  value  from  the 
magnitude  of  the  cross-correlation  (i.e.,  the  amplitude  where  st  and  Sj  have  the  best  match).  For  finite 
duration  scattered  time  signals,  the  frequency-domain  Parseval’s  theorem  may  have  limits. 


SlhlO 

Slhl2 

S4hl0 

S4hl2 

ClhlO 

Clhl2 

C4hl0 

C4hl2 

SlhlO 

1.000 

1.000 

0.902 

0.908 

0.900 

0.900 

0.787 

0.806 

Slhl2 

1.000 

0.902 

0.908 

0.900 

0.900 

0.787 

0.806 

S4hl0 

1.000 

0.876 

0.924 

0.924 

0.919 

0.834 

S4hl2 

1.000 

0.916 

0.916 

0.825 

0.919 

ClhlO 

1.000 

1.000 

0.908 

0.928 

Clhl2 

1.000 

0.908 

0.928 

C4hl0 

1.000 

0.860 

C4hl2 

1.000 

Table  5.  Values  estimated  from  Eq.  (5)  where  Eq.  (4)  is  used  for  (i,j).  By  construction,  the  table  is 
symmetric,  so  the  lower  triangular  elements  are  omitted.  The  light  blue  entries  are  a  result  that  direct 
path  align  and  thus  are  re-enforced  through  the  multiplication  of  the  magnitudes. 

In  the  development  of  Tables  1-5,  only  eight  scattered  signals  from  two  targets  were  considered. 
The  scattering  geometry  and  targets  used  for  the  comparison  for  selected  as  these  are  often  use  in 
benchmark  comparisons  (i.e.,  spherical  target  or  finite-length  cylinder  at  broadside).  The  sequel,  the 
metrics  developed  in  Eqs.  (2)  -  (5)  as  well  as  2D  cross-correlation  are  applied  to  slices  of  data  from 
either  TIER-based  SAS  simulations  or  measurements  (i.e.,  TREX13  data).  A  slice  of  data  consists  of 
81  signals  with  a  0.5°  spacing  along  a  linear  SAS  path.  The  signals  are  30  ms  in  duration  and  a  time 
offset  is  chosen  based  on  an  estimate  of  the  earliest  expected  arrival  from  a  roundtrip,  time-of-flight 
calculation. 

WORK  COMPLETED 

A.  Description  of  TREX13  Data: 

The  TREX13  data  were  collected  by  scanning  a  sonar  tower  along  a  straight  42-m  rail.  Targets 
were  placed  at  horizontal  ranges  from  5  to  40  m  in  5  m  increments.  Table  6  contains  the  ranges  where 
a  given  target  had  been  placed  during  TREX13.  For  an  axially  symmetric  target  such  as  a  water- filled, 
155-mm  howitzer  at  a  given  location,  the  target  was  rotated  from  -80°  (tail  towards  the  rail)  to  80° 
(nose  towards  the  rail)  in  20°  increments.  For  a  non-axially  symmetric  target,  a  complete  target 
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rotation  results  in  18  passes  of  the  sonar  tower  along  the  rail  to  collect  SAS  data  sufficient  for  a 
complete  AC  template.  After  matching-filtering  with  a  replica  of  the  transmitted  signal,  pulse- 
compressed  and  calibrated  target  signatures  were  isolated.  Nine  linear  SAS  data  were  combined  into 
pseudo-CSAS  data,  which  required  processing  to  mitigate  range-migration.  The  pseudo-CSAS  data 
are  then  converted  to  an  AC  template.  The  current  research  steps  back  from  the  pseudo-CSAS  data  to 
directly  use  the  isolated  target  signatures.  This  then  yields  711  sets  of  data  for  proud  targets  and  144 
sets  of  data  for  partially  buried  targets.  Only  the  proud  target  data  are  used  here.  For  a  given  run  along 
the  rail,  1600  pings  at  a  2.5  cm  spacing  are  recorded  with  a  100  kHz  sample  rate  for  a  duration  of  80 
ms.  Thus,  the  size  of  a  SAS  data  set  is  1600x8000.  To  make  the  data  manageable,  it  is  decimated  by 
selecting  those  pings  that  correspond  to  target-centered  angular  values  in  the  range  ±20°  in  0.5° 
increments  (i.e.,  81  pings  are  used).  The  data  are  also  reduced  to  a  30  ms  segment  by  setting  a  time 
offset  based  on  a  time  delay  for  closest  approach  to  a  target.  These  restrictions  led  to  a  reduction  in  the 
size  of  data  to  an  81x3000  slice.  Figure  2a  depicts  the  nine  slices  of  data  for  targetl7  in  Table  6, 
which  is  a  proud  2:1  solid  aluminum  cylinder. 


Target 

Range 

Target 

Range 

Target 

Range 

Target 

Range 

tgtOl 

35 

tgt09 

10,  15,  20,  35 

tgt  16 

15,25,30 

tgt23 

5,  15,  20 

tgt02 

35 

tgtlO 

20, 25, 30 

tgt  17 

10 

tgt24 

5,  15,  20 

tgt03 

35 

tgtll 

10,  20,  25,  35 

tgt  18 

15 

tgt25 

5,  15,  20,  25 

tgt05 

25,  30, 40 

tgtl2 

15 

tgt  19 

20 

tgt28 

15,25,30 

tgt06 

25,  35, 40 

tgtl3 

15,25,35 

tgt20 

10,  15  30,  40 

tgt29 

15,  20, 35 

tgt07 

30,  35, 40 

tgt  14 

15,  20,  25 

tgt21 

10,  15,25,30 

tgt30 

5,  15,  20 

tgt08 

5,  25,  30,  40 

tgtl5 

15,20,  25 

tgt22 

5,  20,  30,  40 

Table  6.  Target  enumeration  and  the  horizontal  ranges  for  deployment  during  TREX13  are 
displayed. 

B.  Description  of  TIER  Simulations: 

The  target-in-the-environment  response  (TIER)  model  was  used  to  simulate  the  scattering  of  a  1-30 
kHz  chirp  from  the  nine  targets  listed  in  Table  7.  The  targets  were  placed  at  horizontal  ranges  of  5  to 
40  m  in  5  m  increments,  and  at  each  range  the  targets  were  rotated  through  the  same  ±80°  in  20° 
increments.  The  source-receiver-target  geometries  were  set  up  to  replicate  a  ±20°  range  with  the  same 
0.5°  sampling  of  the  slices  of  TREX13  data  shown  in  Figure  2.  Signals  were  then  generated  for  a  30 
ms  time  window,  which  is  also  sampled  at  100  kHz. 

TIER  was  used  in  a  number  of  simulations.  First,  free-field  scattering  from  the  targets  was 
predicted  by  including  only  the  direct  path.  Figure  2b  displays  the  predicted  pulse-compressed 
scattering  from  a  2:1  solid  aluminum  cylinder.  Second,  the  scattering  from  the  same  targets  lying 
proud  on  a  sand  sediment  with  environmental  properties  from  TREX13  conditions  was  simulated. 

This  includes  four  paths:  (1)  the  direct  path,  (2)  two  bistatic  scattering  paths  with  a  single  bounce  off 
the  water-sediment  interface,  and  (3)  a  backscattering  path  that  interacts  with  the  water- sediment 
interface  twice.  Figure  2c  depicts  the  scattering  from  the  same  cylinder.  Third,  the  simulations  were 
performed  for  the  full  target  response  and  for  the  target  response  decomposed  into  a  rigid  response  and 
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an  elastic  response.  For  targets  of  nearly  the  same  size  (e.g.,  sim06  -  sim09  of  Table  7),  their  rigid 
responses  may  be  quite  similar,  but  their  elastic  behavior  may  be  sufficiently  different  to  permit  a 
classifier  to  exploit  the  decomposition  of  the  responses.  In  total,  5184  sets  of  simulated  date  were 
generated  with  the  same  size  as  the  slices  of  TREX13  data  (i.e.,  81x3000). 


Target 

Description  of  target 

simOl 

2:1  solid  aluminum  cylinder 

1  sim02 

water-filled  scuba  tank  without  stem 

sim03 

155-mm  howitzer,  filled  with  air,  end  cap 

sim04 

155-mm  howitzer,  filled  with  water,  end  cap 

sim05 

155-mm  howitzer,  no  end  cap 

sim06 

105-mm  bullet-shaped  artillery  shell,  filled  with  air 

sim07 

105-mm  bullet-shaped  artillery  shell,  filled  with  water 

sim08 

100-mm  solid  aluminum  replica  of  artillery  shell 

sim09 

100-mm  solid  stainless  steel  replica  of  artillery  shell 

Table  7.  Targets  used  in  the  TIER  simulations  for  the  metrics  study. 

C.  Four  Potential  Metrics: 

Four  potential  metrics  were  investigated  for  model-model  comparisons,  and  then  these  metrics 
were  applied  to  model-data  comparisons.  One  metric  utilized  the  2D  cross-correlation  of  slices  of 
data.  The  other  three  metrics  are  based  on  total  power  in  the  element-by-element  matrix  product  of 
two  slices  of  data.  Each  metric  is  defined  in  the  following. 

The  first  metric  is  the  conventional  method  of  extracting  the  maximum  value  from  the  absolute 
value  of  a  2D  cross-correlation  of  slices  of  data.  Let  Sj(t,  0)  and  Sj(t,  0 )  represent  two  slices  in  the 
time  and  target-centered  angular  domain.  Let  SL  (f,  9)  and  5)  (f,  9)  represent  the  2D  Fourier 
transforms  of  and  Sj,  respectively.  Then,  this  metric  can  be  written  as 

max\F21{SiSJ*  }| 

\Si>Sj)cc  I - 

j  max  |  F2“ 1  {Si  S  *}  |  max  |  F2- 1  (5)  SJ  }  | 

where  FT1  { . . . )  is  the  2D  inverse  Fourier  transform  and  SJ  is  the  complex  conjugate  of  Sj.  By 
construction,  one  has  0  <  (Si,Sj)cc  <  1. 

The  remaining  three  metrics  were  constructed  from  the  notion  of  total  power.  Parseval’s  theorem 
is 

n  CO  n  CO 

|s(t)|2dt  =  I  \S{f)\2df 

J  —  OO  J—  CO 
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which  states  that  the  total  power  can  be  computed  in  either  the  time  domain  or  the  frequency  domain. 
Let  Sj  o  sj  represent  the  element-by-element  matrix  multiplication  of  the  slices  st  and  Sj .  The  first 
metric  based  on  total  power  can  be  written  as 


(si,sj)1  = 


_ Z  Sj  °  Sj _ 

vz  si  °  si  z  sj  °  sj 


where  £  °  Sj  represents  the  summation  of  the  elements  of  the  matrix.  The  normalization  is  again 

such  that  0  <  (s^  Sj)1  <  1.  For  real-valued  data,  when  i  —  j,  the  numerator  in  Sj)1  is  essentially  a 
discrete  representation  of  the  left-hand-side  of  Parseval’s  theorem  (i.e.,  |s|2  =  s2)  averaged  over  an 
angular  swath.  That  is,  the  summation  over  time  is  a  measure  of  total  power  and  the  summation  over 
angular  swath  gives  the  total  power  in  the  slice  of  data.  When  the  slices  of  data  are  different,  then 
elements  of  st  °  Sj  may  be  negative  and  in  fact  the  summation  may  be  negative. 


For  the  right-hand-side  of  Parseval’s  theorem,  the  real- valued  data  are  Fourier  transformed  from 
the  time  domain  into  the  frequency  domain.  This  results  in  complex-valued  signals  and  one  has  |S|2  = 
|S||S|.  Let  15(1  o  \Sj  |  represent  the  element-by-element  multiplication  of  the  absolute  values  of  the 
elements  of  5,-  and  Sj.  The  second  metric  becomes 

,  a 

JZISlMSilZlSyPISyl 

with  the  summation  indicating  that  all  elements  are  to  be  summed  (i.e.,  an  angular  average  of  a 
frequency-domain  estimate  of  the  total  power  in  a  slice). 


The  third  and  final  metric  utilized  complex-valued  time  signals.  Let  q  be  the  complex-valued 
signal  associated  with  sL.  q  may  be  obtained  from  a  Hilbert  transform,  which  is  often  applied  to 
generate  the  envelop  of  a  time  signal.  This  metric  has  the  form 


(Si.Sj)  3  = 


M 

n 

o 

Cj 

Jlkil  °  |qlZ|tj 

l  0 

Cj\ 

As  in  previous  metrics,  the  summation  in  (Sj,  Sj)3  represents  a  summation  over  both  time  and  angle. 
This  is  then  again  a  measure  of  the  total  power  in  a  slice. 

D.  Model-Model  Comparisons 

When  comparing  the  predictions  of  models,  it  is  assumed  that  the  same  scattering  scenario  has 
been  modeled  (i.e.,  the  same  environment,  target,  and  scattering  geometry).  The  comparison  can  be 
between  distinct  independent  models  (e.g.,  TIER  and  PC-SWAT)  or  a  single  model  executed  with 
limits  on  the  included  physics.  In  the  former,  one  is  trying  to  assess  whether  the  independent  models 
capture  the  same  observables.  That  is,  the  target  signature  produced  by  one  model  should  largely  agree 
with  the  target  signature  produced  the  other.  In  the  latter,  one  is  often  seeking  to  isolate  the  physics 
associated  with  an  observable  (e.g.,  rigid  response  versus  elastic  response).  In  the  following 
comparisons,  the  TIER  model  is  used  with  various  limits  on  the  included  physics. 
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The  first  model-model  comparison  used  the  full  target  response  of  those  listed  in  Table  7.  Figure  3 
shows  the  four  metrics  for  targets  at  a  10-m  horizontal  range,  where  the  9x9  grid  in  each  panel 
delineates  the  nine  target  rotations.  Figure  3a  corresponds  to  the  2D  cross-correlation  metric,  which  by 
its  very  nature  forces  the  “best”  alignment  of  the  data.  The  main  diagonal  is  the  auto-correlation  of  st. 
The  panels  b  -  d  are  for  the  metrics  associated  with  total  power,  and  here  the  main  diagonal  is  similar 
to  the  auto-correlation  in  that  is  compared  to  itself.  It  is  found  that  each  panel  contains  prominent 
sub-diagonals.  These  sub-diagonals  reveal  that  a  slice  of  data  for  one  target  appears  to  be  the  same  as 
a  slice  from  a  different  target. 

The  second  model-model  comparison  involved  the  free-field  scattering  from  a  target  versus  a 
proud  target.  That  is,  the  TIER  model  included  only  the  direct  path  for  free-field  scattering  response; 
while  it  included  the  direct  path  and  3  paths  associated  with  interaction  with  the  water-sediment 
interface  for  a  proud  target.  The  targets  of  Table  7  were  placed  at  a  25  m  horizontal  range  from  a 
linear  SAS  path.  Inspection  of  Figure  4  reveals  that  inclusion  of  only  the  direct  path  captures  some  of 
the  response  for  a  proud  target  (i.e.,  prominent  diagonal  lines).  This  might  be  anticipated  as  the  bottom 
interacting  paths  in  the  proud  response  are  delayed  in  time  with  respect  to  the  direct  path  and  only 
interfere  with  the  direct  path’s  trailing  contribution.  Figure  4  suggests  that  (Sj,  s;  )3  may  provide  better 
discrimination  of  the  targets  than  the  other  metrics. 

The  third  model-model  comparison  decomposed  the  full  target  response  into  a  rigid  component  and 
an  elastic  component.  The  rigid  component  is  typically  associated  with  specular/geometrical 
scattering,  and  an  elastic  response  is  associated  with  elastic  modes  of  oscillation  of  the  target.  The  four 
metrics  are  displayed  in  Figure  5. 

E.  Model-Data  Comparisons 

The  four  metrics  were  applied  to  the  TREX13  data  listed  in  Table  6  and  TIER  simulated  target 
responses  for  targets  in  Table  7.  Examples  of  the  metrics  are  displayed  in  Figure  6  for  targets  at  a  10 
m  horizontal  range.  For  these  comparisons,  the  TREX13  targets  were  tgt09,  tgtl  1,  tgtl7,  tgt20,  and 
tgt21;  while  TIER  simulations  for  all  targets  listed  in  Table  7  were  performed.  Inspection  of  Figure  6 
suggests  that  the  conventional  2D  cross-correlation  metric,  (Sj,  Sj)cc,  performs  better  than  the  other 
metrics  although  (Sj,  Sj)3  does  exhibit  some  sub-diagonals  in  the  TIER  versus  TREX  data  regions. 

This  result  may  be  expected  as  (sL,  Sj)cc  by  construction  aligns  the  data  and  model  result  and  selects 
only  the  maximum  value  in  the  magnitude  of  the  slices;  while  the  other  metrics  implicit  assume  an 
alignment. 

IMPACT/APPLICATIONS 

Four  metrics  were  investigated.  The  metric,  based  on  2D  cross-correlation,  is  typically  used  in 
classification  algorithms.  Model-model  comparisons  suggest  the  other  metrics,  based  on  total  power, 
have  a  potential  to  provide  alternative  computational  kernels  for  classifiers.  Applying  the  same  four 
metrics  to  the  model-data  comparisons  demonstrate  that  uncertainty  in  the  experimental  data 
necessitates  an  alignment  of  data  in  both  the  time  and  target-centered  angle  domains. 

For  n  slices  of  data,  ( Sj ,  Sj  )cc  requires  n!  +  n  2D  Fourier  transforms  to  be  computed.  The 
frequency-domain  total  power  metric,  (Sj,  Sj)2,  requires  only  n  Fourier  transforms  of  the  time  domain 
data.  The  time-domain  metric,  (Sj,s;  )3,  requires  2 n  Fourier  transforms  to  obtain  complex-valued 
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signals.  Finally,  the  simplest  metric,  (si,Sj)1,  requires  only  an  element-by-element  matrix 
multiplication,  which  translates  to  a  significant  reduction  of  computational  cost. 

RELATED  PROJECTS 

The  Performance  Estimate  (PE)  program  is  a  multi-institutional  effort  with  participants  from  APL- 
UW,  ARL-PSU,  GTRI,  and  NSWC-PC.  The  goal  of  this  effort  is  to  establish  procedures  to  assess  the 
overall  performance  of  a  classification  chain.  Although  the  TIER  model  has  been  used  to  generate 
nearly  2  TB  of  data  for  the  PE  program,  the  metrics  investigated  here  have  yet  to  be  integrated  into  a 
(RVM  or  SVM)  classifier. 


Figure  1.  The  lower  four  signals 
correspond  to  targets  at  a  10  m 
horizontal  range,  and  the  upper  four 
signals  are  for  targets  12  m  horizontal 
range.  The  graph  legend  indicates  the 
TIER  simulations  that  contain  only  the 
direct  path  (i.e.,  free-field  scattering) 
and  the  four  ray  paths. 
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Figure  3.  The  results  of  the  four  metrics  applied  to  the  full  target  response  with  the  targets  at  a  10  m 
horizontal  range,  (a)  (Sj,  Sj)cc,  (b)  (sit  Sj )1;  (sit  Sj)2,  and  (d)  (sit  Sj)3 .  The  01-09  labeling  above  each 
coincides  with  simOl -sim09. 
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Figure  4:  Model-model  comparison  of  scattering  from  a  proud  target  (PI 234)  and  a  target  in  the  free 
field  (PI).  The  targets  were  placed  at  a  25  m  horizontal  range.  The  panels  correspond  to  (a) 

(s, > s j ) cc *  (b)  (c)  ( s i t  Sj ) 2 ,  and  (d)  (s^,Sy) 3. 
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Figure  5:  Model-model  comparison  for  a  decomposition  of  the  full  target  response  into  a  rigid 
component  and  an  elastic  component.  The  targets  were  proud  on  a  sand  sediment  interface  at  a  20  m 
horizontal  range.  The  panels  correspond  to  (a)  (Sj,  Sj)cc,  (b)  (sit  Sj )1;  (c)  (Sj,  Sj)2,  and  (d)  (sit  Sj)3 . 
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Figure  6:  Model-data  comparison  for  TREX13  data  and  TIER  model  results.  The  targets  were 
proud  on  a  sand  sediment  interface  at  a  nominal  horizontal  range  of  10  m.  The  panels  correspond  to 
(a)  {sit  Sj)cc,  (b)  ( Si,Sj >1;  (c)  (Si,Sj)2,  and  (d)  (Si,Sj)3. 
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